Stealth Assessment of Problem-Solving Skills from Gameplay

Stealth assessment represents a promising way to address the needs of validly measuring and supporting important 21st century competencies (e.g., creativity, problem solving) within interactive digital environments (e.g., video games). The assessment is woven into the environment such that it becomes invisible to students, which is conducive to eliciting targeted competencies (Shute, 2011). Stealth assessment also runs dynamically, enabling real-time support. We use ECD (Evidence-Centered Design, Almond, & Lukas, 2003) as our assessment design framework for creating stealth assessments that capture far more information related to multiple competencies compared to traditional forms of assessment, which typically report a single summative score, and/or or judgments of right or wrong. To date, we have developed a number of stealth assessments for use in different games to examine various competencies. For example, we have designed three stealth assessments to measure various cognitive and noncognitive variables in a game called Physics Playground (Shute & Ventura, 2013). The focal competencies included persistence (Ventura, Shute, & Small, 2014), qualitative physics knowledge (Shute, Ventura, & Kim, 2013), and creativity (Kim & Shute, in press). From these design and development efforts, we have learned a number of useful lessons about developing and applying stealth assessment. This will comprise the focus of our paper—lessons learned and best practices related to the design process of stealth assessment. We will demonstrate the process of designing stealth assessment using a research project that assesses problem solving skill in the popular game Plants vs. Zombies 2. Results from our evaluation study show that our game-based assessment is promising, correlating with the external measures of problem solving: Raven’s progressive matrices (r = .40, p < .01) and MicroDYN (r = .48, p < .01). However a larger sample size is needed to establish definite claims about its validity.